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ABSTRACT 



To gain insight into whether debate judges actually 
.do treat impromptu speaking as miniextemporaneous speaking, a study 
compared the comments judges wrote to extemporaneous speakers with 
those they wrote to impromptu speakers during the first two rounds of 
a forensic tournament. Approximately 1,000 comments from 152 ballots 
(102 impromptu and 50 extemporaneous) were recorded and broken down 
into the following categories: (1) delivery, (2) specific analysis, 
(content), (3) organization, (4) analysis (general), (5) 
introduction, (6) supporting material, (7*5 conclusion, (8) generally 
positive (related to encouragement for overall performance), (9) time 
allocation, and (10) miscellaneous. Results indicated that judges 
gave students similar if not nearly identical feedback for 
extemporaneous and impromptu speaking Although judges balanced their 
comments among the various criterfra.for effective speech, if it is 
desirable for judges not to treat anHmpromptu event as a 
miniextemporaneous one, then they failed. However, it would be i" 
inappropriate to place all the blame on the judges when the national 
forensic committees have not proposed a clear distinction between the 
two types of events besides- time limits. (JL) 
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In their 1981 study of speech contest judging of poetry 
readings, Janet K. Larsen and JamesCJ. Lewis emphasized the need 
for" consistency among judges for individual events. Bemoaning . 
the Lack of published research in individual events, they' state, 
"Contemporary individual event speech contests are designed to- be 
educational. All too oftenj however, the contest experience ."/ 

• t I 

leaves the participant more confused than educated due to incon-/ 

1.7 >' 

sistent judging criticism." 1 Students -may also become confused/ 
when thecomments they redeive on ballots are inconsistent w*tb 
the way an event is defined by a tournament or forensic associa-j 
tion. , 

This project'not only begiirs to fill the vacuum of .'research 
that exists on the judging of original speaking events!, tout seeks* 
to address the potential inconsistency .a student might Jace when 
judges' comments conflict with rules., 2 In the limited /prepara-'' 
tion , events, most i tournaments try to afist/inguish impromptu speak- 
ing from extemporaneous speaking beyond the differing /time con- 
straints. For example, while ^he American. Forensic Association 
requires the extemporaneous • speaker to address one of "th/pe 
topics in the general area of current events*" it defines [Lm- 
promptu speaking as "an .imprbmptu speech, serious ih nature, with 
topic selection varied -round by round," noting that "Topics shall 
be of a proverb nature." 3 Furthermore, the Nationall Forensic 

; / 

.Association rules stress that impromptu "is not mi/ni-extemp." 4 
.If impromptu speaking is not mini-ex terrfp as specified in the 
• rules of- both major forensic associations, then it' should *in some 
way be judged' differently from extemporaneous spejaking. To gain* f 
insight into whether or not judges actually do tr'eat impromptu as 



nrini-extemp, % the present study seeks to answer the research ques- 
tion "To wl\at extent are the comments judges write to extempo- , 
raneous speakers similar to those they write to impromptu speak- 



ers?" 




Other issUes that must be explored in order to answer the \ 

> 

central question inciifde the ' following: What types of comments 

do judges make to speakers in each event? How many comments of 
« 

each ty-pe'per ballot do judges make? Of all of the comments made 
to a speaker in a particular event, what percentages N of comments 
fall into each category? Do the proportions of comments in cat- 

* V 

egor^es follow similar patterns in the two events? 
/. " 

To answer the above questions, 1,048 comments from 152 bal- 
lots were analyzed. The sai/ple of ballots was taken from the 
finst^&tfo rounds of extemp and impromptu at ttfe Cornhusker Foren- 
sicj|||frnament held at the University of Nebraska in Lincoln 
F^yi§Pf ry ; 25-26, 1983. As 51 impromptu and 25 extemporaneous . 

w 

stants entered the tournament, the sample contained 102 
rqmptu ballots^nd 50 extemporaneous ballots. 

Upon reading the ballots, the number of comments were 



piounted. A comment, is defined any sentence, phrase, or single 
word that provides some critique of the speaker's performance or 
advice for impfoven\ent. For example, "I don't think you can pro- 
vide a strong -analysis if you give a non-cqmmi ttal answer" and* 
$food! n both were counted as single comments because they both 
relayed a single message. On some occasions, a sentence con- 



we*,.*:*... 



tained two clauses revealing different messages. For example, 
the sentence "You've talked about Reagan's 'control over the is- 
sue, but you should have focused on exactly what Reagan has done 
and what he hasn't done" was coded as two comments. While it ^ 

praised the speaker for addressing the issue in the first clause 
v 

it suggested the contestant improve how the issue was addressed i 
in the second. However, sentences such ^s "Because of your humor, 
I .was at ease" would be classified as one comment. 

The comments were therx broken down into content categories. 
An inductive method was used to generate these categories, with 
new categories created each time a comment appeared that did not 
appear in a previously encountered classification. The cat- 
egories into which 95 per cent or more of the comments fell were 
used for statistical analysis, with the rej^t of the comments 
being placeld into a miscellaneous category. The categories # of 
comments post^analy tically derived from the sample are defined as 
follows: 

i 

1. Delivery — any comment addressing 1 any issue related to either 
physical or vocal delivery. Examples: "Your overall delivery 
was outstanding." "Good eye contact." "You had a pleasant 
voice." "The pace of your speech was refreshing." 

2. Specific Analysis (Content) — Any comment addressing ££J2li£= 
itllL the quality of the way in which a, speaker handled the 
contept of 'the speech. Examples: "I didn't buy . the sexist 
implication^ at the end at all," "Any ideas on the impact of 

1 that deprivation," and "I think you would have done well to ■ 
discuss Adelman's possible appointment to the negotiating 
team," and "How does- Reagan's budget cuts affect me?" NOT 
INCLUDED: "I was rather confused, by your analysis throughout 
the speech." 



* 3. Organisation — Any comment related to the structure of the 
student's speech. -Examples: - ■ "You might have placed the 
backgjfound- information ^arlier in the speech," "Good transi- 
tionj^rom the introduction to the body, of your ,speech," and 
"a lot of mjr confusion centered around your organization." 
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'M A nalysis (General)— Any comment assessing the student's anal- 
ysis but not refering to the specific content of the speech. 
"Your overall analysis was good" would fall into this cat- 

> egory," along with "More concrete analysis needed earlier in 
your speech" and 'the like. 

5. Introduction — Any* comment, that focuses on the_introductionoi»f 
the speech. Examples: "Your intro was outstanding, " "Good 
preview of your main points," and "Nice attention-getter." 

6. Supporting Material— Any comment questioning or applauding 
the student's sources and examples. Examples: "Good sources 
on the current economic problems," "You rely too much on U.S. 

t News," and "You- use many good . personal examples here." 

7. Conclusion— Any comment- related to the student's conclusion 
of a speech or closure: Examples: "Nice referring back to 
your introductory remarks in the conclusion'"- and "A summary 
at the end would be helpful." 

8; Generally positive— Any comment that o'ffers encouragement to 
the student's o.y_e_r.all performance as opposed to a specific 
aspect of performance. Usually the last comment on a go^od 
ballot, although it can be use\to encourage anyone. Examples: 
"You are a marvelous speaker," "Good show!" and- "Stick with 
it." 

9. Time allocation — Any comment that refers to the way a student 
used his or her time either before or while speaking^ Ex- 
amples: "You need to use more of your prep time," and "You 
need to spend more time on the second point." 

Miscellaneous — Any comment not focusing on any of the above* 
"reas. Examples: . "Good word usage" (3 comments appeared on 
word usage), "Good humor" (3 comments appeared on humor) and 
"I wish you had stayed around for questions." (Comments on 
» the question perKiod of extemp appeared 5 time's) 



AsjQhe ballots were read and comments counted, all of the 
data were broken down further by event fbr comparative purposes. 
Two statistical ^nalyses were performed on the data to determine 
to what extent the judges were making similar comments to con- 
testants from one event to the other. First, the category totals 
for the two events were correlated using Pearson's r to determine 
the overall strength 1 * of the similarity in comment type from one 
event to the other. Second, for 'each comment category, the mean 




number of comments per ballot for extemporaneous ballots was 
compared, to the mean number of comments per ballot for impromptu 
using the ttesf for statistical significance. The second measure 
enabO.es us to ascertain whether j udges" pla.ce more emphasis on a 
particular judging criterion in either extemporaneous or impromp- 
tu speaiking. 

Ee.s_u.it_s. " < 

Analysis of the ballots yielded the following breakdown of 
the 1,048 comments: 



XIEE Ql QSMH31 t 

Delivery 62 

Specific Analysis 79 

Organization 61. 

Analysis (General) 60 

Introduction 51 

Supporting Material 59 

Conclusion 18 

Generally Positive 9 

Time Allocation 1 

Mi__.c_.ellaneo.Lis u 

Total 411 



SYEH1 

Extemp Impnfimptu 
iZb_al A I I __.Zb_al A i 

1.24 15.0 139 1:36 21.8 201 

1.58 19.2 96 .94 15.0 175 

94 .92 14.8 155 

83 . .81 13. 0 C 143 

79 .77. 12.4 130 

61 .60 9.5 120 

37 .36 5.8 • 55 

2.2 16 .16 2.5 25 

0.01 8 .08 1.3 9 

1*3 ZH 3j£ 25 

637 6;24 1048 



1 .22 
1 .20 
1 .02 
1 .18 
.36 
.18 
.02 

8,22 



19.2 
14.8 
14.7 
12.4 
14.3 
4.3^ 



Ifital Sample. 
i iZfeal^ i 
1.32 19.2 
1.15 16.7 
1.02 14.8 
.94 13.6 
.86 12.4 
.79 11.5 
.36 5.2 
.16 2.4 
.06 0.9 

6.89 ' 



The results indicate that judges, are giving the students 
^^milar if not nearly identical feedback f or "exfcemp and impromptu 



speaking. Aside from delivery ^supporting' material } 
analysis, the percentages of comments in- each catego 
are remarkably similar from one event to another 



and specific 
y of comment 
In two cate- v 



gories (organisation and introduction) judges allocated equal 
percentage-j of comments (14.8: per cent for organization and 12.4 
per cent for introduction-), and in yet* another, conclusion, they 
commented the same number of times on the average per ballot 
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From the averages and percentages above, it would appear 
.that judges emphasized delivery slightly more often on their 
impromptu ballots C.1 . 3 6 comments* per ballot, 21.8 per cent of 

\ 

total comments) than they did on their extemporaneous ballots 
(1.24 commments per ba|lot, 15.0 per cent of total comments). On 
the other hand, judges stressed specific analysis and supporting 
material more in extemporaneous speaking. 

While small differences exist in the percentages of comments 
in a few categories, the correlation a£aly si»/indicates the over- 
all strength of "the similarity in comments betwe'en the samples of 
extemporaneous impromptu ballots. The' Pearson correlation 
of the category totals was an extremely high .887, and was sig- 
nificant at the .001 level. 

Although judges tended to comment more often on extempor- 
aneous ballots (8,22 comments^ per ballot) than they did on 
impromptu ^allots (6.24 comments per ballot), only, in two of the 
categories were their average numbers of comments^for extemp 
greater enough than the average numbers for impromptu to attain 
a statistical significance at the .05 level. Even where the 
difference in means was statistically significant (in specific 
analysis where t=2.383 with 150 degrees of freedom and in 
supporting material where t=3.966 wUh 150 degrees of free- 
dom), much of the difference in the number of comments from one 
event to another can probably be explained by the fact that 
overall judges commented more often on extemp 'ballots than they 
did on impromptu ballots. While judges on the average tended to 



ERIC 



a- \ 



\ 



both stress delivery more often on impromptu ballots and devote a 
greater proportion of. their comments on impromptu ballots to 
delivery, the t r test showed 0 that this difference was not statis- 
tically significant. * * 

v ' * 

From the results of* the present study, five issues warrant 

our attention. First, the results show- what judges ar£ ; saying 

most often to the contestants on their ballots. .Second, judges 

x 

at the tournament tended to treat impromptu speaking as mini- 
extemp. Thir<i, the results point to th/.need for judges to ' 
enforce the rules given for limited preparation events/ Fourth, 
the outcomes of the ball^t^analy sis must be'taken in light of this 
study's limitations. Fifth, while the present study has its « 
limitations-, it has strengths that warrant further study into the 
area of judging\criteria. , 

If it is considered desirable for judges to balance their 
comments among the various criteria for effective speech and to 
stress analysis and supporting' material in extemp and imprdmptu, 
then the judges whose ballots were studied performed admirably. 
No one^category contained more than 20 per cent of all comments 
,for the total sample, and for the event samples, only the one ' • 
^category contained over that percentage, (delivery in impromptu, 
21.8 per cent) Six categories of comments on extemp ballots and 
five categories on impromptu ballots each contained over 10 pet 
cent of the comments within their respective samples. In terms 
of stressing analysis and supporting materials, the judges made 
more than 42 per cent of their comments in these areas. 



However, if it is desirable for judges not to treat'' 
impromptu as a mini-extemp event, then the judges "failed" miser- 
ably. The" extremely .high Pearson correlation and it's .• • 
significance indicated that the judges are treating impromptu as 
a mini-extemp event, contrary ,to the rules established by the 
national forensic associations. . Nonetheless, no-t all q% the } * 
blame for- the inconsistency .between feedback &nd rules can be 
placed on the judges. While both national tournaments suggest 
that extemporaneous ^and imprompt'u events' 'differ along lines that 
go beyond the .differing time constraints, neither tournament 
(which influences rules at every tournament wishing to be a ' 
qualifier) spells out what that difference is. N.F.A's rule ' 
stating that in im-promptu "contestants will receiv^short ex- 
cerpts on general interest, political/ economic, an^social 
issues" 'belies its admonition that impromptu "is not mini- 
extemp." And while A.F.A stipulates that its impromptu topics be 
of a "proyerb" rather than current events nature, none- of' its 
rules spell .out any difference in how the student is to ad&nesa a 

certain topic. * 

_ 

The national committees therefore ne-ed tojudecide first whe- 
ther there need be any distinction betjween the events 'other than 
time constraints. If they decrde that there need be no 
•distinction but for time, Jbhen the rules should reflect that 
similarity, which the judges now seem to be "enforcing." ' Other- 
wise, should the committees decide that greater distinction 
should be made, they should then make explicit these distinctions 
in the^rules so that the judges migh.t know what they are enfor- 
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cipg', if there is supposed to be any distinction in' the way. a ' 
-.student approaches the two events, it would- appear that judges 
are not now recognizing' that distinction. ' With clearer guide- 
lines, the judges would be better able give feedback to students, 
more/consistent with the rules -those students are taught. 

These implications should be taken in light of the following 
limitations. First; the sample was taken from one tournament and 
might therefore reflect a regional, bia-s. Second, the baliots 
comprising the sample came from a limited number of judges. 
Third, the method" of »averaging did not eliminate or account for 
extreme cases. Howeyer, the number of comments studied was lafge 
enough to mimimize the .extreme case problem. Finally, the 
• results are limited in th.at not all judges reflect in their 
•comments to contestants all of the reasons*-- or even the important 
reasons — for their decisions. 

On the other hand, 'the ^results are strenghtened by three ma- 
jor factors. First, as two ballots for each alot in these events 
were studied, the cases came - from ballots written to a wide range 
of students' running the gamut of both speaking style and talent. 
'Second, the categories were post-ana"ly tically derived". Finally,, 
the extreme strength of the correlation between samples adds to 
the power of the findings. 

From the baliots studied, then, there would appear to be a 
very strong similarity between 'the criteria, judges use to assess 
extemp and impromptu speeches. Aside from discovering this ap- 
parent similarity, this study has contributed classif icstory and 

• ^ • • ' « 

comparatory systems that might be useful in future study of 

-i 

judging criteria for public speaking events. Particularly inter- 



esting in terms of preparing students for nationals wou,ld be a ' 
study of possible regional biases in judging. In preparing' stu- 
dents for the transition from high school to collegiate compe-' 
tition, a study comparing crfteria used by high schpttfand col- 
legiate judges would .be useful. Finally, projects such as the 
Larson and Lewis study of consistency between judges in inter- 
pr t etive speaking events could be carried but with respect to the 
public speaking events. Results of such studies could be useful 
if we are to leave our students more educated than confused as a 
result of their participation in forensics. 



A 



■<? 10 



12 



1 , . - * 

.. James J,. Lewis, and Janet K. Larsen, * "Inter-rater Judge 
.^Agreement in Forensic 'Competition," J_Q.u_r.nal fif the. Allianifian 
I Eoxejasic- As.sacj.atian '18 ( S'umme r 1 9 8 1 ) ; 9 . 

2 A Survey of JJ.EA, QiS* CM*and the regional speech 
journals ' revealed not a .single study even remotely like the 
present study regarding original speaking. events. There were ' 
some interpretation ballot studies among the convention papers: 
surveyed, but none empirical. They were Stacey Cox, "Textual 
Considerations in Prose Interpretation," Bruce B. Manchester, 
"Teaching D.I.: Textual Considerations," and Bob Frank, ^Com- 
petitive Interpretation of Poetry: Rhetoric yersus Poetic," each 
presented at the 1981 Speech Communication Association annual. * 
convention at Anaheim. 

J American Forensic Association National Individual Events 
Tournament Invitation, 1981. 

< • ' ■> 

National Forensic Association National Individual Events 
Trournament Invitation, 1981, * 



QJS - Quarterly Journal of Speech 
CM - Communication Monographs 
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